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Technological  advances  in  helmet-mounted  displays  (HMDs)  have  permitted  the  design 
of  “see-through”  displays  in  which  virtual  imagery  may  be  superimposed  upon  real  visual 
environments.  The  utility  of  see-through  displays  in  multitask  environments  remains  un¬ 
certain,  especially  in  environments  that  involve  switching  one’s  attention  between  those 
tasks  represented  in  the  virtual  display  and  those  existing  in  the  real  world.  The  present 
study  was  designed  to  assess  the  effects  of  a  secondary  visual  monitoring  task  on  per¬ 
formance  and  workload  in  a  head-slaved  tracking  task.  Participants  attempted  to  center  a 
reticle  over  a  moving  circular  target  using  a  Kaiser  Electronics  SimEye  2500  HMD  while 
concurrently  performing  the  visual  monitoring  task  component  of  the  Multi-Attribute 
Task  Battery  (MATB;  Comstock  &  Arnegard,  1992),  which  was  displayed  on  a  computer 
monitor.  Task  difficulty  for  the  head-slaved  tracking  task  was  varied  by  manipulating 
time  delay.  Results  are  discussed  in  terms  of  their  implications  for  practical  implementa¬ 
tion  of  see-through  HMDs  in  multi-task  environments. 


INTRODUCTION 

Advances  in  helmet-mounted  display  (HMD)  technology 
have  permitted  the  design  of  “see-through”  displays  in  which 
virtual  imagery  may  be  superimposed  upon  real  visual  envi¬ 
ronments.  Indeed,  see-through  HMDs  have  numerous  poten¬ 
tial  applications  ranging  from  augmented  displays  for  teleop- 
erated  surgery  (Durlach  &  Mavor,  1995)  to  head-slaved  dis¬ 
plays  for  tactical  aviation  (Beal  &  Sweetman,  1994).  In  the 
case  of  the  latter,  see-through  HMDs  may  afford  the  design  of 
head-slaved  Head-Up  Displays  (HUDs),  thereby  permitting  the 
display  of  flight-critical  information  regardless  of  where  the 
pilot  is  looking.  In  addition,  when  see-through  HMDs  are 
used  in  conjunction  with  helmet-mounted  sights  and  high  off- 
boresight  weapon  systems,  pilots  are  provided  with  the  unique 
tactical  advantage  of  designating  targets  that  are  up  to  90°  off 
the  nose  of  the  aircraft. 

Notwithstanding  their  potential  to  enhance  human  percep¬ 
tion  and  performance  in  complex  task  environments,  see- 
through  displays  are  confronted  by  many  technological  chal¬ 
lenges,  including  misalignment  of  virtual  imagery  with  real 
world  objects  (Azuma  &  Bishop,  1994),  optical  distortion  and 
glare,  and  problems  generic  to  most  HMDs  (e.g.,  helmet  fit 
and  discomfort,  field  of  view  limitations,  suboptimal  resolu¬ 
tion,  and  issues  involving  time  delay).  In  the  case  of  time  de¬ 
lay,  Ricard  (1994)  noted  the  accumulation  of  a  sizable  litera¬ 
ture  demonstrating  the  deleterious  effects  of  time-delayed  vis¬ 
ual  feedback  on  an  operator’s  ability  to  manually  control  and 
regulate  dynamic  systems.  These  effects  also  seem  to  be  pres¬ 
ent  in  tasks  involving  head-slaved  tracking.  For  example.  Nel¬ 


son  and  his  colleagues  (1998)  recently  showed  that  the  addi¬ 
tion  of  67  ms  of  time  delay  significantly  degraded  performance 
efficiency  in  a  head-slaved  tracking  task  using  a  non-see- 
through  HMD.  Similar  results  have  been  reported  by  So  and 
his  colleagues  (So,  Chung,  &  Goonetilleke,  1999;  So  &  Grif¬ 
fin,  1991). 

While  empirical  investigations  of  the  so-called  time  delay 
problem  in  HMDs  have  been  abundant,  these  studies  have 
typically  been  conducted  in  single-task  environments,  e.g., 
tracking  tasks,  in  spite  of  the  facts  that  1)  one  of  the  principal 
advantages  of  see-through  HMDs  is  their  ability  to  support 
operators  in  multi-task  environments;  and  2)  attentional  limi¬ 
tations  in  dual-task  performance  are  well  documented  (Pashler 
&  Johnston,  1998).  As  an  example  of  the  former,  it  may  prove 
utile  for  operators  to  track  and  designate  targets  using  a  see- 
through  HMD,  while  concurrently  monitoring  control  station 
displays  in  their  immediate  workspace.  Accordingly,  the  pur¬ 
pose  of  the  present  investigation  was  to  assess  the  effects  of 
both  time  delay  and  a  secondary  visual  monitoring  task  on 
performance  efficiency  and  operator  workload  in  a  head- 
slaved  visual  tracking  task. 

METHOD 

Participants 

Seven  naive  participants,  4  females  and  3  males,  served  in  the 
experiment.  Their  ages  ranged  from  20  to  32  years  with  a 
mean  of  24.35  years.  Participants  reported  normal  or  cor- 
rected-to-normal  vision,  and  indicated  that  they  were  not 
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highly  susceptible  to  motion  sickness.  In  addition,  all  partici¬ 
pants  reported  no  prior  experience  with  head-slaved  tracking 
tasks  using  an  HMD.  Participants  were  paid  for  their  partici¬ 
pation. 

Experimental  Design 

A  within-subjects  design  was  employed  in  which  two  time 
delay  conditions  ( nominal  and  nominal  +  100  ms)  were  com¬ 
bined  with  two  task  conditions  ( single  and  dual)  and  five  ex¬ 
perimental  sessions.  The  single  task  condition  required  only 
the  performance  of  the  head-slaved  tracking  task,  while  the 
dual  task  required  participants  to  perform  the  tracking  task  and 
the  monitoring  task  concurrently.  The  order  of  the  time  delay 
condition  was  randomized  across  participants,  while  the  order 
of  the  task  conditions  was  fixed  within  each  session  (i.e., 
blocks  of  single-task  trials  preceded  the  dual-task  trials). 

Apparatus  and  Procedure 

Each  experimental  session  included  20  5-min  head-slaved 
tracking  trials.  The  first  10  trials  served  as  a  baseline  condi¬ 
tion  for  head-slaved  tracking  performance  and  did  not  require 
the  participant  to  perform  the  visual  monitoring  task.  Trials 
11-20  involved  both  the  tracking  and  monitoring  tasks.  Prior 
to  the  initiation  of  the  main  experimental  sessions,  all  partici¬ 
pants  completed  five  5-min  practice  trials  of  the  secondary 
visual  monitoring  task.  The  purpose  of  the  practice  trials  was 
to  acquaint  participants  with  the  response  procedures  for  the 
task  and  to  ensure  that  they  were  able  to  perform  the  task  at 
ceiling  level. 

Participants  used  a  Kaiser  Electronics  SimEye  2500  HMD 
to  track  a  moving  visual  target.  The  SimEye  2500  HMD  em¬ 
ploys  an  optical  relay  system  to  transfer  video  images  from  a 
pair  of  green  phosphor  monochrome  cathode  ray  tubes  (CRTs) 
to  the  participant's  eyes.  It  features  a  high  resolution  (1280  x 
1024  pixels)  binocular  display  and  was  configured  to  provide 
subjects  with  a  60°  (horizontal)  x  40°  (vertical)  field  of  view 
(FOV).  The  optical  focus  range  of  the  SimEye  2500  extends 
from  3.5  feet  to  infinity,  and  was  set  to  infinity  in  the  present 
experiment.  The  SimEye  2500  weighs  approximately  four 
pounds  and  was  configured  as  a  see-through  display,  thereby 
allowing  participants  to  view  the  visual  display  on  which  the 
monitoring  task  was  presented. 

The  head-slaved  tracking  task  employed  target  motion 
patterns,  or  forcing  functions,  that  consisted  of  the  sum  of 
three  sine  waves  with  fundamental  frequencies  of  0.067,  0.1 17, 
and  0.233  Hz  in  azimuth,  and  0.083,  0.167,  and  0.217  Hz  in 
elevation.  Target  motions  were  restricted  to  ±30°  in  azimuth, 
and  +20°  in  elevation.  Different  target  motions  were  gener¬ 
ated  for  each  trial  by  randomly  assigning  phase  values  at  each 
of  the  three  fundamental  frequencies  in  azimuth  and  elevation. 

Head  position  and  orientation  were  measured  by  an  As¬ 
cension  Bird  tracker.  The  Bird  consists  of  a  DC  magnetic- 
field  transmitter  and  a  receiver  that  was  mounted  atop  the 
HMD.  The  Bird  provides  six  degrees-of-freedom  tracking  at 
120  Hz  while  minimizing  interference  caused  by  nearby  me¬ 
tallic  objects.  All  phases  of  the  head-slaved  tracking  task  and 


data  collection  were  governed  by  a  200  MHz  personal  com¬ 
puter.  Target  and  head  position  data  were  collected  at  60  Hz 
for  each  5-min  trial. 

The  nominal  time  delay  in  the  head-slaved  tracking  sys¬ 
tem  was  determined  to  be  46  ms.  The  imposed  time  delay 
consisted  of  six  frames  of  delay,  or  100  ms  (six  frames  @  16.7 
ms).  Thus,  in  the  time  delayed  condition,  the  total  time  delay 
of  the  system  was  approximately  146  ms. 

The  secondary  monitoring  task  consisted  of  the  systems 
monitoring  task  from  the  Multi-Attribute  Task  Battery 
(MATB;  Comstock  &  Arnegard,  1992).  In  short,  the  task 
comprised  a  set  of  four  gauges  with  moving  pointers.  Under 
non-signal  conditions,  the  moving  pointers  oscillated  around 
the  center  tickmark  by  no  more  than  one  mark  from  the  center 
tickmark  on  each  of  the  gauges.  A  critical  signal  consisted  of 
any  of  the  four  pointers  moving  more  than  one  mark  from  the 
center  of  the  gauge  in  either  direction.  Participants  were  in¬ 
structed  to  inspect  the  gauges  for  critical  signals  and  to  make 
the  appropriate  keyboard  response  as  soon  as  one  was  de¬ 
tected.  Critical  signals  not  detected  within  10  s  were  scored  as 
missed  signals;  conversely,  responses  to  non-signals  were 
scored  as  errors  of  commission.  The  MATB  monitoring  task 
also  included  a  pair  of  system  status  displays  positioned  above 
the  four  gauges.  The  normal  or  non-signal  condition  for  these 
displays  were  the  presence  of  a  green  light  on  the  left  display 
and  a  black  fill  on  the  right  display.  Critical  signals  consisted 
of  the  left  display  shifting  from  green  to  black,  or  the  right 
display  shifting  from  black  to  red.  Again,  participants  were 
instructed  to  inspect  the  system  status  display  for  critical  sig¬ 
nals  and  to  respond  as  soon  they  detected  a  change  in  system 
status.  During  each  of  the  5-min  experimental  trials  12  critical 
signals  were  presented  -  two  critical  signals  for  each  of  the 
four  gauges  and  two  critical  signals  for  each  of  the  system 
status  displays. 


SVSTEM  MONITORING 


Figure  1.  Participant  performing  head-slaved  tracking  task  and  secondary 
visual  monitoring  task. 


Upon  arrival,  participants  were  presented  with  an  over¬ 
view  of  the  experimental  procedure,  received  instructions,  and 
donned  the  HMD.  Proper  fit  and  viewing  quality  in  the  HMD 
were  achieved  by  making  adjustments  to  its  inter-pupillary 
distance  controls,  vertical,  tilt,  and  axial  helmet  angles,  chin 
strap,  variable-thickness  foam  pads,  and  inflatable  air-bladder. 
Participants  completed  20  5-min  head-slaved  tracking  trials 
per  experimental  session  -  ten  trials  with  and  without  the  ad¬ 
ditional  visual  monitoring  task  (see  Figure  1).  Each  5-min  trial 
was  preceded  by  a  5  s  target  acquisition  period  to  ensure  that 
participants  had  acquired  the  target  at  the  onset  of  the  trial. 


Tracking  performance,  however,  was  based  on  the  5-min  trial 
and  did  not  include  the  5  s  acquisition  period.  Participants 
completed  the  NASA  Task  Load  Index  at  the  completion  of  5- 
min  trial  and  received  a  10-min  rest  period  after  the  comple¬ 
tion  of  five  experimental  trials. 

RESULTS 

Head-slaved  Tracking  Performance 

For  each  5-min  head-slaved  tracking  trial,  target  and  head 
position  data  were  used  to  calculate  two  indices  of  error  in  the 
time  domain:  percent  time  on  target  (TOT)  and  root  mean 
squared  (RMS)  error.  The  latter  provides  an  unsigned  meas¬ 
ure  of  error  between  the  center  of  the  aiming  reticle  and  the 
center  of  the  visual  target;  the  former  provides  a  measure  of 
the  percentage  of  time  that  the  center  of  the  aiming  reticle  is 
within  the  boundary  of  the  visual  target. 

Time  on  Target.  Mean  TOT  percentages  for  all  experi¬ 
mental  conditions  were  subjected  to  a  2  (time  delay)  x  2  (task) 
x  5  (trials)  repeated  measures  analysis  of  variance,  which  re¬ 
vealed  main  effects  of  time  delay ,  Fi  1,6)  =  9.04,  p  <  .05,  and 
task,  E(l,6)  =  46.37,  p  <.05,  and  a  significant  time  delay  x 
task  interaction,  Fi  1,6)  =  6.66,  p  <  .05.  All  other  sources  of 
variance  lacked  statistical  significance  (i.e.,  p  >  .05).  The  time 
delay  x  task  interaction  is  depicted  in  Figure  2,  which  shows 
the  decrements  in  tracking  performance  associated  with  the 
addition  of  both  time  delay  and  the  visual  monitoring  task. 
The  interaction  can  be  explained  by  noting  that  the  presence  of 
the  additional  time  delay  degraded  tracking  efficiency  to  a 
greater  extent  in  the  tracking-only  condition  than  it  did  when 
participants  were  required  to  perform  the  monitoring  task. 


Task  Condition 

Figure  2.  Mean  percent  time-on-target  associated  with  the  two  time  delay 
conditions  under  each  level  of  the  task  condition. 

Root  Mean  Squared  Error.  Mean  RMS  Errors  were  sub¬ 
mitted  to  a  similar  repeated  measures  ANOVA,  revealing  sig¬ 
nificant  main  effects  of  time  delay,  Fi  1 ,6  )  =  7.12,  p  <  .05  and 
task,  F{  1,6)  =  36.39,  p  <.05,  but  failing  to  disclose  the  signifi¬ 
cant  time  delay  x  task  interaction  revealed  by  the  analysis  in¬ 
volving  the  TOT  data.  The  main  effects  are  illustrated  in  Fig¬ 
ures  3a  and  3b. 


Task  Condition  DelaV  Condition 


Figures  3a  and  3b.  Mean  RMS  error  for  each  task  condition  (a)  and  for  each 
level  of  time  delay  (b). 

Visual  Monitoring  Performance 

Performance  on  the  secondary  visual  monitoring  task  was 
determined  for  each  experimental  session  and  scored  in  terms 
of  correct  detections  (hits)  and  errors  of  commission  (false 
alarms). 

Correct  Detections.  Mean  percentages  of  correct  detec¬ 
tions  for  the  two  time  delay  conditions  across  the  five  experi¬ 
mental  trials  are  presented  in  Table  1.  Inspection  of  the  table 
reveals  that  performance  on  the  secondary  monitoring  task 
approached  ceiling  performance  and  that  performance  was 
rather  consistent  across  the  experimental  conditions.  A  2  (time 
delay)  x  5  (trials)  repeated  measures  ANOVA  of  these  data 
confirmed  these  impressions  by  revealing  no  significant  main 
effects  or  interactions. 

Table  1 

Mean  Percent  Correct  Detections 

Experimental  Trials 

Time  Delay  Condition  1  2  3  4  5  A 

Nominal  97.6  98.8  97.6  97.6  99.9  98.3 

Nominal  +  too  ms  97.6  95.2  96.4  98.8  97.6  97.1 

Errors  of  Commission.  Mean  percentages  of  errors  of 
commission,  or  false  alarms,  are  shown  in  Table  2  for  all  ex¬ 
perimental  conditions.  Review  of  these  data  indicates  that  the 
occurrence  of  false  alarms  varied  across  experimental  condi¬ 
tions;  however,  an  ANOVA  of  these  data  failed  to  reveal  any 
systematic  sources  of  variance. 

Table  2 

Mean  Percent  Errors  of  Commission 

Experimental  Trials 

Time  Delay  Condition  1  2  3  4  5  A 

Nominal  4.8  20.2  8.3  10.7  7.1  10.2 

Nominal  +  100  ms  8.3  15.5  9.5  9.5  4.8  9.5 

Operator  Workload  Ratings 

The  NASA  Task-Load  Index  (NASA-TLX;  Hart  & 
Staveland,  1988),  a  multidimensional  scale  of  perceived  men¬ 
tal  workload,  was  used  to  provide  subjective  estimates  of  the 
information  processing  demands  associated  with  the  experi- 


mental  task.  The  NASA-TLX  provides  a  global  measure  of 
overall  workload  (on  a  scale  of  0  to  100),  and  also  identifies 
the  relative  contributions  of  six  sources  of  workload:  (1) 
Mental  Demand,  (2)  Physical  Demand,  (3)  Temporal  Demand, 
(4)  Performance,  (5)  Effort,  and  (6)  Frustration. 

Overall  Workload  Ratings.  Mean  overall  workload  rat¬ 
ings  were  submitted  to  a  2  (time  delay)  x  2  (task)  x  5  (trials) 
repeated  measures  ANOVA,  revealing  a  significant  main  ef¬ 
fect  of  task ,  F(  1,6)  =  10.42,  p  <  .05,  but  neither  significant 
main  effects  of  time  delay  nor  trial  nor  significant  interactions 
involving  any  of  the  factors.  Mean  overall  workload  scores  for 
the  tracking  and  tracking+monitoring  task  conditions  were 
38.75  and  46.74,  respectively. 

Subscale  Ratings.  Mean  weighted  ratings  for  the  NASA- 
TLX  subscales  were  submitted  to  a  5  (subscale)  x  2  (time  de¬ 
lay)  x  2  (task)  x  5  (trials)  repeated  measures  ANOVA. 
Weighted  ratings  for  the  Frustration  subscale  contributed  least 
to  ratings  of  workload  and  were  subsequently  excluded  from 
the  ANOVA  in  order  to  meet  the  independence  assumption  of 
the  analysis.  Significant  sources  of  variance  resulting  from  the 
analysis  included  a  main  effect  of  task,  F(  1,6)  =  8.39,  p  <  .05 
and  a  significant  task  x  subscale  interaction,  F( 4,24)  =  2.87,  p 
<  .05.  All  other  sources  of  variance  were  not  significant;  how¬ 
ever,  it  is  worth  noting  that  the  effects  of  subscale  and  time 
delay  were  associated  with  /;- values  of  .055  and  .053,  respec¬ 
tively.  The  task  x  subscale  interaction  is  shown  in  Figure  4,  in 
which  mean  weighted  subscale  ratings  are  plotted  for  the 
tracking  and  tracking+monitoring  task  conditions  across  the 
six  subscale  dimensions.  Post  hoc  pairwise  comparisons  (Bon- 
ferroni-adjusted  f-tests)  of  the  task  conditions  at  each  subscale 
revealed  that  the  addition  of  a  secondary  monitoring  task  re¬ 
sulted  in  significantly  greater  ratings  of  Mental  Demand,  but 
failed  to  show  differences  for  the  other  subscales. 


Effort  Performance  Mental  Temporal  Physical  Frustration 
Demand  Demand  Demand 
Workload  Subscale 


Figure  4.  Mean  weighted  subscale  ratings  (NASA-TLX)  under  each  level  of 
the  task  condition. 

CONCLUSION 

The  present  study  represents  an  initial  effort  to  evaluate 
the  effects  of  a  secondary  monitoring  task  on  head-slaved 
tracking  performance  and  workload  using  a  see-through  HMD. 
Additionally,  the  effects  of  time  delay  on  head-slaved  tracking 


were  investigated  in  the  context  of  a  dual-task  environment, 
thereby  extending  the  work  in  this  area  (see  Nelson  et  al., 
1998;  So  &  Griffin,  1991).  While  this  experiment  was  con¬ 
ducted  in  a  reasonably  controlled  laboratory  setting,  the  find¬ 
ings  reported  herein  are  anticipated  to  generalize  to  more  ap¬ 
plied  situations  which  incorporate  see-through  HMDs  in  dual¬ 
task  environments. 

Hitherto,  time  delay  in  HMD  systems  has  been  regarded 
as  one  of  the  principal  constraints  on  their  functional  utility, 
especially  in  tightly-coupled  control  tasks.  Yet  the  data  pre¬ 
sented  here  compel  one  to  conclude  that  its  effects  -  on  work¬ 
load  as  well  as  performance  -  may  be  rendered  inconsequen¬ 
tial  by  the  introduction  of  a  secondary  task  of  even  modest 
complexity.  Such  an  outcome  may  be  of  import  to  human 
factors  professionals  who  prematurely  recommend  the  adop¬ 
tion  of  see-though  HMDs  as  a  solution  to  problems  inherent  in 
multi-task  work  domains. 
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